The traditional photographic process utilizes analog film and is guided by a relatively fixed paradigm: capture, develop, and print. Analog film has pretty much given way to digital sensors, and consequently services that once required expertise and darkrooms can now be performed by the broader population using readily accessible post-processing software.
Stack-based photography aims at overcoming the limitations of standard digital cameras by taking a sequence (stack) of images with different capture parameters (e.g., different focus distances and/or exposure times), and then combining those images into a single image that “best” represents the image stack. Thus, the traditional workflow has largely remained the same: capture, perform some optional processing, and produce a single final image that is perhaps printed.
The human visual system does not, however, see everything in a real-world scene at once. Rather, the eyes move over the scene adapting for the amount of light and required focus at each location. At any given time, for instance, the eyes can only handle a dynamic range of approximately 1:100, and the perception of the scene is not impacted by the fact that objects outside of the current region of interest saturate to black or white. Similar considerations apply to focus and to color perception, which depend on the point of attention and on the spatial distance of other colors from the point of attention.
Also, the tendency today is to rarely if ever print pictures. Instead, pictures are viewed on digital screens: computer monitors, cameras, smart phones, tablets, or other potentially interactive display surfaces. Yet, despite the additional degrees of freedom that digital displays offer and the aforementioned tendencies of the human visual system, much of the time only a single static image, conventionally generated from a stack of images as described above, is displayed.
Accordingly, most computational photography techniques are still designed to enhance the capture capabilities of modern cameras in the context of the static print paradigm: create an image that captures and visualizes a larger portion of the dynamic range, or create an image with a larger depth of field. Accomplishing these goals often involves hardware in the form of exotic sensor architectures or unusual optics, as is the case for high dynamic range (HDR) imaging or all-in-focus imaging. Stack-based photography achieves similar results with standard camera hardware. However, most of these approaches do not capitalize on the additional degrees of freedom available when pictures are viewed on display monitors.
Additionally, some existing techniques that are used to reduce image stacks to a single picture demand pixel-accurate registration in order to avoid ghosting and misalignment artifacts that can negate the benefits that might be achieved by capturing multiple images. Even when pixel-accurate registration is possible, compositing techniques generally must compromise on one aspect of image quality in order to improve another. For example, when an HDR image is tone-mapped to a low dynamic range (LDR) image suitable for print or conventional displays, either the local or the global contrast, often both, has to be lowered.